fix: switch canterbury to the council's new bin dates endpoint - #2308

Canterbury’s scraper now requests collection dates from a bins-widget API. It handles date data provided as an object or JSON string, maps seven collection types, and sorts the formatted results.

Changes: Canterbury scraper

Layer / File(s)	Summary
API request and date parsing uk_bin_collection/uk_bin_collection/councils/CanterburyCityCouncil.py	The scraper posts the UPRN to the bins-widget API with browser-like headers and a 30-second timeout. It parses dates provided as an object or JSON string, maps seven collection types, formats and sorts results, and raises errors for HTTP failures or missing collections.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: m26dvd

Merge Risk: Low · up to 26928

Normal lookups are not shown to be affected, but an unexpected API response would fail with an unhandled error. This is a bounded risk that can be addressed with response-shape checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title names the main change: Canterbury's bin-collection scraper now uses the council's new bin dates endpoint. The raw summary and PR objectives confirm this. The title is specific and concise, a…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Title check

Explanation

The title names the main change: Canterbury's bin-collection scraper now uses the council's new bin dates endpoint. The raw summary and PR objectives confirm this. The title is specific and concise, and it fits the changeset. The title also uses the conventional "fix:" prefix, which fits a change that restores working behaviour after the old endpoint returned 403 for every request.

🧪 Generate unit tests (beta)

• Create a new PR

• X
• Mastodon
• Reddit
• LinkedIn

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[CanterburyCityCouncil]

Flake rate in main: 99.70% (Passed 2 times, Failed 664 times)

Stack Traces | 0.204s run time

fixturefunc = <function scrape_step at 0x7c57f199d220>
request = <FixtureRequest for <Function test_scenario_outline[CanterburyCityCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7c57f18ff380>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
                           ^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/collect_data.py:109: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:130: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:71: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:94: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../uk_bin_collection/councils/CanterburyCityCouncil.py:50: in parse_data
    response.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [403]>

    def raise_for_status(self) -> None:
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://n6ljrw455m.execute-api.eu-west-2.amazonaws.com/prod/get-bin-dates

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/requests/models.py:1167: HTTPError